{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### clone代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/modelscope/modelscope-agent.git"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 安装特定依赖"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd modelscope-agent && pip install -r requirements.txt\n",
    "!pip install vllm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 本地配置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.chdir('modelscope-agent/demo')\n",
    "\n",
    "import sys\n",
    "sys.path.append('../')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 本地部署llm\n",
    "modelscope提供模型本地启动服务功能。这里我们使用该功能，将模型部署成openai api兼容的接口。具体操作可参考如下：\n",
    "#### 下载模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -c \"from modelscope import snapshot_download; model_dir = snapshot_download('qwen/Qwen1.5-7B-Chat', cache_dir='qwen1.5-7b-chat');print(model_dir)\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 部署模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 在终端里执行下面的命令启动llm模型服务\n",
    "!MODELSCOPE_CACHE='qwen1.5-7b-chat' python -m vllm.entrypoints.openai.api_server \\\n",
    "    --model qwen/Qwen1.5-7B-Chat --dtype=half --max-model-len 8192  --gpu-memory-utilization 0.95"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 测试部署\n",
    "测试模型服务，如果正确返回，说明模型服务部署完成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 在终端里执行下面的命令，测试模型服务\n",
    "!curl http://localhost:8000/v1/chat/completions \\\n",
    "    -H \"Content-Type: application/json\" \\\n",
    "    -d '{\n",
    "        \"model\": \"qwen/Qwen1.5-7B-Chat\",\n",
    "        \"messages\": [\n",
    "            {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
    "            {\"role\": \"user\", \"content\": \"写一篇春天为主题的作文\"}\n",
    "        ],\n",
    "        \"stop\": [\"<|im_end|>\", \"<|endoftext|>\"]\n",
    "    }'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from modelscope_agent.agents import RolePlay\n",
    "\n",
    "role_template = '你是一个agent小助手，你需要根据用户的要求来回答他们的问题'\n",
    "llm_config = {\n",
    "    'model': 'qwen/Qwen1.5-7B-Chat', \n",
    "    'model_server': 'openai',\n",
    "    'api_base':'http://127.0.0.1:8000/v1',\n",
    "    'api_key': 'EMPTY'\n",
    "    }\n",
    "function_list = []\n",
    "\n",
    "bot = RolePlay(function_list=function_list,llm=llm_config, instruction=role_template)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 执行agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = bot.run(\"你好，请以李云龙的语气和我对话\")\n",
    "text = ''\n",
    "for chunk in response:\n",
    "    text += chunk\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "agentfabric也支持本地部署llm，若您需要使用，可以查看文档：[docs/local_deploy.md](../docs/local_deploy.md)"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
